Methods and Systems for 360-DEGREE video post-production

ABSTRACT

In accordance with example embodiments, the method and system for 360-degree video post-production generally makes use of points of view (POVs) to facilitate the 360-degree video post-production process. A POV is a rectilinear subset view of a 360-degree composition based on a particular focal length, angle of view, and orientation for each frame of the 360-degree composition. Video post-production editing can be applied to a POV by the user, using rectilinear video post-production methods or systems. The rectilinear video post-production editing done on the POV is integrated back into the 360-degree environment of the 360-degree composition. In accordance with example embodiments, the method and system for 360-degree video post-production comprises identifying a point of view in a 360-degree composition; applying video post-production editing to the point of view; and aligning a new layer containing the video post-production editing with the point of view in the 360-degree composition.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a Continuation-In-Part of U.S. non-provisionalapplication Ser. No. 15/588,073, filed May 5, 2017, which has issued asU.S. Pat. No. 9,959,905 on May 1, 2018, the contents of which are hereinincorporated by reference.

TECHNICAL FIELD

Example embodiments relate generally to virtual reality and augmentedreality, and in particular for video post-production for virtual realityand augmented reality.

BACKGROUND

Video post-production refers to editing of videos following the shootingor generation of videos. Video post-production can include visualeffects, video stabilization, modification (editing) of objects, andinsertion of objects.

Video post-production of 360-degree videos has introduced a number ofnew challenges, which make it difficult to use some existing tools,which were developed for post-production of 2D rectilinear videos.

Additional difficulties with existing systems may be appreciated in viewof the Detailed Description of Example Embodiments, herein below.

SUMMARY

In accordance with an example embodiment is a method and system for360-degree video post-production generally makes use of points of view(POVs) to facilitate the 360-degree video post-production process. A POVis a rectilinear subset view of a 360-degree composition based on aparticular focal length, angle of view, and orientation for each frameof the 360-degree composition. Video post-production editing can beapplied to a POV by the user, using rectilinear video post-productionmethods or systems. The rectilinear video post-production editing doneon the POV is integrated back into the 360-degree environment of the360-degree composition.

In accordance with an example embodiment, the method and system for360-degree video post-production generally includes identifying a pointof view in a 360-degree composition; applying video post-productionediting to the point of view; and aligning a new layer containing thevideo post-production editing with the point of view in the 360-degreecomposition.

In accordance with an example embodiment, the method and system for360-degree video post-production generally includes identifying a pointof view in a 360-degree composition; applying tracking to the point ofview to identify tracking information consisting of one or more pointcloud layers and a tracked camera layer; linking the point cloud layerto the tracked camera layer; and aligning the tracked camera layer withthe point of view in the 360-degree composition.

In accordance with an example embodiment, the method for 360-degreevideo post-production, the method being performed by at least oneprocessor, generally includes identifying a point of view comprising anidentified rectilinear view of a 360-degree composition; applying videopost-production editing to the point of view; creating one or morelayers containing the video post-production editing; and aligning theone or more layers with the identified point of view in the 360-degreecomposition.

In accordance with an example embodiment, the method for 360-degreevideo post-production further includes repeating the method ofidentifying a point of view comprising an identified rectilinear view ofa 360-degree composition; applying video post-production editing to thepoint of view; creating one or more layers containing the videopost-production editing; and aligning the one or more layers with thepoint of view in the 360-degree composition with one or more same ordifferent points of view.

In accordance with an example embodiment, the video post-productionediting comprises importing media of a media file aligned with theidentified point of view. In accordance with an example embodiment, themedia file is a 360-degree media file. In accordance with an exampleembodiment, the video post-production editing on the point of viewcomprises tracking the imported media from the point of view to identifytracking information comprising one or more point cloud layers and atracked camera layer; and linking the one or more point cloud layers tothe tracked camera layer.

In accordance with an example embodiment, the video post-productionediting on the point of view comprises tracking the point of view toidentify tracking information comprising one or more point cloud layersand a tracked camera layer; and linking the one or more point cloudlayers to the tracked camera layer.

In accordance with an example embodiment, the method for 360-degreevideo post-production further comprises orienting the 360-degreecomposition with the inversion of the orientation of the tracked cameralayer; and updating the orientation of the identified point of view tomatch the orientation of the tracked camera layer after the trackedcamera layer is aligned with the identified point of view.

In accordance with an example embodiment, the method for 360-degreevideo post-production further comprises selecting one or more points ofview in addition to the identified point of view; calculating thedifference in orientation between the one or more selected points ofview and the identified point of view before the identified point ofview is updated to match the orientation of the tracked camera layer;and updating the orientation of the one or more selected points of viewto match the addition of the calculated difference and the orientationof the tracked camera layer after the tracked camera layer is alignedwith the identified point of view.

In accordance with an example embodiment, the method for 360-degreevideo post-production further comprises a memory for storing the360-degree composition.

In accordance with an example embodiment, the method for 360-degreevideo post-production further comprises displaying an interface screenfor the 360-degree composition, the interface screen configured todisplay an omni directional camera viewer that is configured to displaya rectilinear view of the 360-degree composition as a consequence toorientation navigating through the omni directional camera viewer; andreceiving selection of the displayed rectilinear view, wherein theidentified rectilinear view is the selected displayed rectilinear view.In accordance with an example embodiment, said orientation navigatingand/or said receiving selection are controlled using an input interfacedevice. In accordance with an example embodiment, the displayedrectilinear view is also the consequence of navigating one or more ofthe angle of view and focal length.

In accordance with an example embodiment, the displayed rectilinear viewis generated by identifying and displaying a subset of the 360-degreecomposition corresponding to the displayed rectilinear view

In accordance with an example embodiment, the identified point of viewincludes a particular focal length, angle of view, and orientation ofthe 360-degree composition.

In accordance with an example embodiment, the identified point of viewis selectable from any viewing angle. In accordance with an exampleembodiment, the identified point of view is off-angle from a rectilinearviewing angle.

In accordance with an example embodiment, the method for 360-degreevideo post-production further comprises rendering a new 360-degree videofile which renders the 360-degree composition together with the one ormore layers. In accordance with an example embodiment, the new360-degree video is aligned with the identified point of view or adifferent point of view. In accordance with an example embodiment, thenew 360-degree video file is stored in memory or sent to another device,server, or cloud server.

In accordance with an example embodiment, the video post-productionediting comprises generating a sub-composition of the identified pointof view and applying rectilinear post-production editing to only thesub-composition.

In accordance with an example embodiment, the sub-composition of theidentified point of view is generated by flattening all layers in the360-degree composition into a single sub-composition layer.

In accordance with an example embodiment, the video post-productionediting comprises adding a new solid color layer or adjustment layersized to overlay the identified point of view.

In accordance with an example embodiment, the system for 360-degreevideo post-production comprises memory; an output interface devicecomprising a display; an input interface device; a processor operablycoupled to the memory, the output interface device and the inputinterface device, the processor for executing instructions stored in thememory which, when executed, causes the processor to perform a method of360-degree video post-production.

In accordance with an example embodiment, the system for 360-degreevideo post-production comprises a non-transitory computer readablemedium comprising instructions for 360-degree video post-production andexecutable by one or more processors, the instructions comprisinginstructions for performing a method of 360-degree videopost-production.

Another example embodiment is a method for 360-degree videopost-production, the method being performed by at least one processor,the method including:

displaying an interface screen for a 360-degree composition on an outputinterface device comprising a display, the interface screen configuredto display an omni directional camera viewer that is configured todisplay a rectilinear view of the 360-degree composition as aconsequence to orientation navigating through the omni directionalcamera viewer; identifying a point of view comprising receivingselection through the interface screen of a rectilinear view displayedin the interface screen; applying video post-production editing to theidentified point of view, comprising tracking the identified point ofview to identify tracking information comprising one or more point cloudlayers and a tracked camera layer; linking the one or more point cloudlayers to the tracked camera layer; aligning the one or more point cloudlayers with the identified point of view in the 360-degree composition;identifying a further point of view comprising receiving selectionthrough the interface screen of a further rectilinear view displayed inthe interface screen; said applying video post-production editingfurther being applied to the further point of view and furthercomprising tracking the further point of view to identify secondarytracking information comprising one or more secondary point cloud layersand a secondary tracked camera layer; linking the one or more secondarypoint cloud layers to the secondary tracked camera layer; aligning thesecondary tracked camera layer with the identified further point ofview; scaling position parameters of the secondary tracked camera layerto match position parameters of the tracked camera layer; and saving oneor more layers, comprising the points cloud layers and the trackedcamera layer, to memory.

In accordance with an example embodiment, the method further comprises:identifying an existing or additional point of view; said applying videopost-production editing comprising video post-production editing of theidentified existing or additional point of view to create one or moreadditional layers; aligning the one or more additional layers with theidentified existing or additional point of view in the 360-degreecomposition; and saving the one or more additional layers to memory.

In accordance with an example embodiment of the method, said applyingvideo post-production editing comprises importing media of a media file.

In accordance with an example embodiment of the method, the media fileis a 360-degree media file of 360-degree footage taken from a 360-degreecamera.

In accordance with an example embodiment, the method further comprises:orienting the 360-degree composition with an inversion of an orientationof the tracked camera layer; and updating an orientation of theidentified point of view to match the orientation of the tracked cameralayer after the tracked camera layer is aligned with the identifiedpoint of view.

In accordance with an example embodiment, the method further comprises:selecting one or more points of view in addition to the identified pointof view; calculating a difference in orientation between the selectedone or more points of view and the identified point of view before theorientation of the identified point of view is updated to match theorientation of the tracked camera layer; and updating the orientation ofthe one or more selected points of view to match the addition of thecalculated difference and the orientation of the tracked camera layerafter the tracked camera layer is aligned with the identified point ofview.

In accordance with an example embodiment of the method, said orientationnavigating and/or said receiving selection are controlled using an inputinterface device.

In accordance with an example embodiment of the method, the displayedrectilinear view is also the consequence of navigating one or more ofthe angle of view and focal length.

In accordance with an example embodiment of the method, the displayedrectilinear view is generated by identifying and displaying a subset ofthe 360-degree composition corresponding to the displayed rectilinearview.

In accordance with an example embodiment of the method, the identifiedpoint of view includes a particular focal length, angle of view, andorientation of the 360-degree composition.

In accordance with an example embodiment of the method, the identifiedpoint of view is selectable from any viewing angle.

In accordance with an example embodiment, the method, further comprisesrendering a new 360-degree video file which renders the 360-degreecomposition together with the one or more layers, and saving the new360-degree video file to the memory.

In accordance with an example embodiment of the method, the new360-degree video file is rendered by flattening all of the one or morelayers in the 360-degree composition.

In accordance with an example embodiment of the method, the videopost-production editing comprises: generating a sub-composition of theidentified existing or additional point of view; and said applying videopost-production editing comprises applying rectilinear post-productionediting to the sub-composition.

In accordance with an example embodiment of the method, said scalingfurther comprises scaling the one or more secondary point cloud layersbased on said secondary tracked camera layer position parameters.

In accordance with an example embodiment, the method further comprisesstabilizing the 360-degree composition based on the trackinginformation.

In accordance with an example embodiment, the method further comprisesoutputting the 360-degree composition together with the one or morelayers to a 360-degree display.

In accordance with another example embodiment is a system for 360-degreevideo post-production, comprising: memory; an output interface devicecomprising a display; an input interface device; a processor operablycoupled to the memory, the output interface device and the inputinterface device, the processor for executing instructions stored in thememory which, when executed, causes the processor to perform: displayingan interface screen for a 360-degree composition on an output interfacedevice comprising a display, the interface screen configured to displayan omni directional camera viewer that is configured to display arectilinear view of the 360-degree composition as a consequence toorientation navigating through the omni directional camera viewer;identifying a point of view comprising receiving selection through theinterface screen of a rectilinear view displayed in the interfacescreen; applying video post-production editing to the identified pointof view, comprising tracking the identified point of view to identifytracking information comprising one or more point cloud layers and atracked camera layer; linking the one or more point cloud layers to thetracked camera layer; aligning the one or more point cloud layers withthe identified point of view in the 360-degree composition; identifyinga further point of view comprising receiving selection through theinterface screen of a further rectilinear view displayed in theinterface screen; said applying video post-production editing beingapplied to the further point of view and further comprising tracking thefurther point of view to identify secondary tracking informationcomprising one or more secondary point cloud layers and a secondarytracked camera layer; linking the one or more secondary point cloudlayers to the secondary tracked camera layer; aligning the secondarytracked camera layer with the identified further point of view; scalingposition parameters of the secondary tracked camera layer to matchposition parameters of the tracked camera layer; and saving one or morelayers, comprising the points cloud layers and the tracked camera layer,to memory.

In accordance with another example embodiment is a non-transitorycomputer readable medium comprising instructions for 360-degree videopost-production and executable by one or more processors, theinstructions comprising instructions to perform: displaying an interfacescreen for a 360-degree composition on an output interface devicecomprising a display, the interface screen configured to display an omnidirectional camera viewer that is configured to display a rectilinearview of the 360-degree composition as a consequence to orientationnavigating through the omni directional camera viewer; identifying apoint of view comprising receiving selection through the interfacescreen of a rectilinear view displayed in the interface screen;

applying video post-production editing to the identified point of view,comprising tracking the identified point of view to identify trackinginformation comprising one or more point cloud layers and a trackedcamera layer; linking the one or more point cloud layers to the trackedcamera layer; aligning the one or more point cloud layers with theidentified point of view in the 360-degree composition; identifying afurther point of view comprising receiving selection through theinterface screen of a further rectilinear view displayed in theinterface screen; said applying video post-production editing beingapplied to the further point of view and further comprising tracking thefurther point of view to identify secondary tracking informationcomprising one or more secondary point cloud layers and a secondarytracked camera layer; linking the one or more secondary point cloudlayers to the secondary tracked camera layer; aligning the secondarytracked camera layer with the identified further point of view; scalingposition parameters of the secondary tracked camera layer positionparameters to match position parameters of the tracked camera layer; andsaving one or more layers, comprising the points cloud layers and thetracked camera layer, to memory.

In accordance with another example embodiment is a method for 360-degreevideo post-production, the method being performed by at least oneprocessor, the method comprising: displaying an interface screen for a360-degree composition on an output interface device comprising adisplay, the interface screen configured to display an omni directionalcamera viewer that is configured to display a rectilinear view of the360-degree composition as a consequence to orientation navigatingthrough the omni directional camera viewer; identifying a point of viewcomprising receiving selection through the interface screen of arectilinear view displayed in the interface screen;

applying video post-production editing to the identified point of view,comprising tracking the identified point of view to identify trackinginformation comprising one or more point cloud layers and a trackedcamera layer; linking the one or more point cloud layers to the trackedcamera layer; aligning the one or more point cloud layers in the360-degree composition with the identified point of view; and saving oneor more layers, comprising the point cloud layers and the tracked cameralayer, to memory.

In accordance with an example embodiment, the method further comprises:identifying an existing or additional point of view through theinterface screen; said applying video post-production editing comprisingvideo post-production editing of the identified existing or additionalpoint of view to create one or more additional layers; aligning the oneor more additional layers in the 360-degree composition with theidentified existing or additional point of view; and saving the one ormore additional layers to memory.

In accordance with an example embodiment, said applying videopost-production editing comprises importing media of a media file.

In accordance with an example embodiment the media file is a 360-degreemedia file of 360-degree footage taken from a 360-degree camera.

In accordance with an example embodiment, the method further comprisesstabilizing the 360-degree composition based on the trackinginformation.

In accordance with an example embodiment, said stabilizing furthercomprises: orienting the 360-degree composition with an inversion of anorientation of the tracked camera layer; and updating an orientation ofthe identified point of view to match the orientation of the trackedcamera layer after the tracked camera layer is aligned with theidentified point of view.

In accordance with an example embodiment said stabilizing furthercomprises: selecting one or more points of view in addition to theidentified point of view; calculating a difference in orientationbetween the selected one or more points of view and the identified pointof view before the orientation of the identified point of view isupdated to match the orientation of the tracked camera layer; andupdating the orientation of the one or more selected points of view tomatch the addition of the calculated difference and the orientation ofthe tracked camera layer after the tracked camera layer is aligned withthe identified point of view.

In accordance with an example embodiment, said orientation navigatingand/or said receiving selection are controlled using an input interfacedevice.

In accordance with an example embodiment, the displayed rectilinear viewis also the consequence of navigating one or more of the angle of viewand focal length.

In accordance with an example embodiment, the displayed rectilinear viewis generated by identifying and displaying a subset of the 360-degreecomposition corresponding to the displayed rectilinear view.

In accordance with an example embodiment, the identified point of viewincludes a particular focal length, angle of view, and orientation ofthe 360-degree composition.

In accordance with an example embodiment, the identified point of viewis selectable from any viewing angle.

In accordance with an example embodiment, the method further comprisesrendering a new 360-degree video file which renders the 360-degreecomposition together with the one or more layers, and saving the new360-degree video file to the memory.

In accordance with an example embodiment, the new 360-degree video fileis rendered by flattening all of the one or more layers in the360-degree composition.

In accordance with an example embodiment, the video post-productionediting comprises: generating a sub-composition of the identifiedexisting or additional point of view; and said applying videopost-production editing comprises applying rectilinear post-productionediting to the sub-composition.

In accordance with an example embodiment, the video post-productionediting comprises rectilinear post-production editing.

In accordance with an example embodiment, the method further comprisesoutputting the 360-degree composition together with the one or morelayers to a display.

In accordance with an example embodiment, the display is a 360-degreedisplay.

In accordance with another example embodiment is a system for 360-degreevideo post-production, comprising: memory; an output interface devicecomprising a display; an input interface device; a processor operablycoupled to the memory, the output interface device and the inputinterface device, the processor for executing instructions stored in thememory which, when executed, causes the processor to perform: displayingan interface screen for a 360-degree composition on an output interfacedevice comprising a display, the interface screen configured to displayan omni directional camera viewer that is configured to display arectilinear view of the 360-degree composition as a consequence toorientation navigating through the omni directional camera viewer;identifying a point of view comprising receiving selection through theinterface screen of a rectilinear view displayed in the interfacescreen; applying video post-production editing to the identified pointof view, comprising tracking the identified point of view to identifytracking information comprising one or more point cloud layers and atracked camera layer; linking the one or more point cloud layers to thetracked camera layer; aligning the one or more point cloud layers in the360-degree composition with the identified point of view; and saving oneor more layers, comprising the point cloud layers and the tracked cameralayer, to memory.

In accordance with another example embodiment is a non-transitorycomputer readable medium comprising instructions for 360-degree videopost-production and executable by one or more processors, theinstructions comprising instructions to perform: displaying an interfacescreen for a 360-degree composition on an output interface devicecomprising a display, the interface screen configured to display an omnidirectional camera viewer that is configured to display a rectilinearview of the 360-degree composition as a consequence to orientationnavigating through the omni directional camera viewer; identifying apoint of view comprising receiving selection through the interfacescreen of a rectilinear view displayed in the interface screen; applyingvideo post-production editing to the identified point of view,comprising tracking the identified point of view to identify trackinginformation comprising one or more point cloud layers and a trackedcamera layer; linking the one or more point cloud layers to the trackedcamera layer; aligning the one or more point cloud layers in the360-degree composition with the identified point of view; and saving oneor more layers, comprising the point cloud layers and the tracked cameralayer, to memory.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made, by way of example, to the accompanyingdrawings which show example embodiments of the present application, andin which:

FIG. 1 shows a block diagram illustrating a 360-degree videopost-production system in accordance with an example embodiment;

FIG. 2 shows a flow diagram illustrating an example method for360-degree video post-production in accordance with an exampleembodiment;

FIG. 3 is an illustration of a Point-of-View (POV) in a 360-degreevideo;

FIG. 4 is a screen shot illustrating a project tab on an example userinterface screen displayed on the video post-production system of FIG.1.

FIG. 5 is a screen shot illustrating a 360 tab on an example userinterface screen displayed on the video post-production system of FIG.1.

FIG. 6 is a screen shot illustrating a Picture-In-Picture (PIP) tab onan example user interface screen displayed on the video post-productionsystem of FIG. 1.

FIG. 7 is a screen shot illustrating the POV tab on an example userinterface screen displayed on the video post-production system of FIG.1.

FIG. 8 is a screen shot illustrating a tracking tab on an example userinterface screen displayed on the video post-production system of FIG.1.

FIG. 9 is a screen shot showing an example of an equirectangular map ofa frame of a 360-degree video.

FIG. 10 is a screen shot showing an example of a cube map of a frame ofa 360-degree video.

FIG. 11 is a screen shot illustrating an example omni directional cameraviewer interface.

FIG. 12 is a flow diagram of a method for 360-degree videopost-production, in accordance with an example embodiment.

FIG. 13 is a flow diagram of a method for importing tracking informationinto a 360-degree video post-production project, in accordance with anexample embodiment.

FIG. 14 illustrates a 360-degree display that is a head-mounted display,to which example embodiments can be applied.

Similar reference numerals may have been used in different figures todenote similar components.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

A number of tools have been developed to assist in video post-productionof rectilinear videos. For example, there are a number of differenttools that assist with adding visual effects into rectilinear videos.Visual effects include compositing of videos or stills, addition ofcomputer generated imagery, color correction, and other manipulations ofthe video as originally shot. Generally, visual effects can be used toadd or remove aspects to a video after the original shot. An example ofvideo composition is the addition of an actor from a video filmed in agreen screen or blue screen environment into a different video. Computergenerated imagery, which can include animated objects, can also be addedinto a video. Some example tools for adding visual effects intorectilinear videos include Adobe™ After Effects™, Maxon Cinema4D™, TheFoundry™ Nuke™, Autodesk™ 3D Studio Max™, Autodesk™ Maya™, SideFX™Houdini™, and Blender™.

One method of assisting in video post-production is tracking. Motiontracking can be used to track scenes and objects in videos to assist inadding visual effects that follow or are affected by the scene or anobject. By tracking an object, the tracking tools are able to determineor at least approximate an object's position and orientation relative tothe camera. Objects in a video can be tracked in two dimensions or threedimensions. Tools that assist in tracking for videos include Adobe™After Effects™ PFTrack™, Boujou™, 3D Equilizer™ and Syntheyes™.

Tracking can also be used to solve the camera. When solving a camera,typically a stationary object or objects in the video are tracked, todetermine the position and orientation relative to the camera throughoutthe video. Since the object is known to be stationary, any changes inposition and orientation relative to the camera can be understood to bea result of camera movement. This information can be used to solve thecamera, meaning the tracking tool will have identified the camera'sposition, focal length, angle of view and orientation throughout thevideo. A solved camera can be used to stabilize footage, for example bycompensating for any changes in camera orientation. A solved camera canalso be used to assist in adding visual effects to a specific locationwithin a scene.

There has been a growing interest in 360-degree videos. Unlikerectilinear videos, 360-degree video recordings record in multipledirections or every direction at the same time. 360-degree videos areoften viewed using a head-mounted display, which allows a viewer to lookaround in all directions while watching the video. 360-degree videos canalso be viewed using 360-degree projectors or on a display that allowsthe user to change the view direction.

There are a number of map types, also referred to as projections, whichare used for recording a 360-degree video or image or displaying a360-degree video or image in a flat, non-360-degree environment. Commonexamples are the equirectangular map 900, an example of which is shownin FIG. 9, which projects the 360-degree video onto a flat rectangularview; and the cube map 1000, an example of which is shown in FIG. 10,which projects the 360-degree video onto 6 square panels that representthe six inside faces of a cube which, when being viewed in a 360-degreeenvironment, are arranged to surround the video viewer's perspective.There are different arrangements of the 6 square panels of a cube map1000, for example the H-cross or V-cross.

In accordance with example embodiments, the method and system for360-degree video post-production generally makes use of points of view(POVs) to facilitate the 360-degree video post-production process. A POVis a rectilinear subset view of a 360-degree composition based on aparticular focal length, angle of view, and orientation for each frameof the 360-degree composition. Referring is made to FIG. 12, whichprovides a flowchart of basic steps in a method for 360-degree videopost production 1200 in accordance with an example embodiment. Inaccordance with an example embodiment, a POV is identified 1202 and thenvideo post-production editing is applied to the POV 1204 by the user,using rectilinear video post-production methods or systems. Therectilinear video post-production editing done on the POV is integratedback into the 360-degree environment of the 360-degree composition. Inaccordance with an example embodiment, a new layer is created 1206 whichincludes the rectilinear post-production editing. In accordance with anexample embodiment, the new layer is aligned with the identified POV1208.

In accordance with at least some an example embodiment, rectilineartracking methods or systems can be used on a POV. The rectilineartracking information is then applied in the 360-degree context of the360-degree composition. In accordance with an example embodiment, thetracking done on a particular POV is used to solve the camera for the360-degree composition. In accordance with an example embodiment, theorientation of the 360-degree camera can be used to stabilize the360-degree composition. A stabilized 360-degree composition can becreated by canceling out the orientation changes of the 360-degreecamera.

Reference is made to FIG. 1 which shows a block diagram illustrating a360-degree video post-production system 101 in accordance with anexample embodiment. In accordance with an example embodiment, the system101 can be implemented on a computer, the system 101 can be a customizedsystem dedicated to 360-degree video post-production, or the system 101may be implemented in a distributed manner, with parts of the system 101implemented on different physical devices, such as those skilled in theart will understand is the case in cloud computing environments.

In accordance with an example embodiment, the system 101 includes acontroller comprising at least one processor 140 (such as amicroprocessor) which controls the overall operation of the system 101.The processor 140 interacts with device subsystems which may include oneor more displays (screens) 104 which can be a computer monitor such as aliquid crystal display (LCD) screen or other displays such as atelevision screen or projector; input devices 106 such as a keyboard,graphic tablet or mouse; one or more data storage devices 142; randomaccess memory (RAM) 146; and read only memory (ROM) 148. In accordancewith an example embodiment, the data storage devices 142 can anycomputer readable medium or a combination of different computer readablemediums. In accordance with an example embodiment, the system 101 mayinclude one or more graphic processing units (GPU) 141 in addition tothe at least one processor 140. A GPU 141 may have its own volatilememory, such as RAM. One or more of the displays 104 may interactdirectly with the GPU 141. In accordance with an example embodiment, thesystem 101 may also include one or more auxiliary input/output (I/O)subsystems 150 and one or more data ports 152 such as a serial dataport, such as a Universal Serial Bus (USB) data port.

In accordance with an example embodiment, the auxiliary input/output(I/O) subsystems 150 and data ports 152 can be used to connectperipherals to the system 101. In accordance with an example embodiment,the auxiliary input/output (I/O) subsystems 150 may comprise an externalcommunication link or interface, for example, an Ethernet connection.The auxiliary I/O subsystems 150 or data ports 152 may also be used toconnect removable memory to the system 101.

In accordance with an example embodiment, the system 101 may alsoinclude one or more speakers 156; one or more microphones 158; and otherdevice subsystems generally designated as 164. In accordance with anexample embodiment, the speakers 156, microphones 158, or other devicesubsystems 164 may be connected directly to the processor 140 or may beconnected through an auxiliary I/O subsystem 150 or a data port 152.

In accordance with an example embodiment, the speaker 156 may comprisespeakers configured in stereo or multi-directional, and the microphone158 may be stereo or multi-directional. For example, real-worlddirectional sound can be recorded using a multi-directional microphone158. When a 360 video is played, the associated sound can then be outputfrom the speaker 156 from the recorded sound, and/or other soundtracksand/or sound that is edited, added, or removed, for example.

In accordance with an example embodiment, the system 101 can include oneor more digital cameras 160. The camera(s) 160 may be separable from orintegrated within the system 101. Camera(s) 160 include a lensconfiguration and associated software application(s) for detecting andcapturing real-world images or video. Integrated cameras can storereal-world images or videos into a data storage device 142 or RAM 146.Separable cameras 160 may have internal memory (not shown) for storingreal-world images or videos. In accordance with an example embodiment,when a separable camera 160 is reconnected to the system 101, forexample through an I/O subsystem 150 or serial port 152, the real-worldimage or video files stored on the camera's 160 internal memory can bedownloaded into the system's data storage devices 142 or RAM 146.

In accordance with an example embodiment, one or more of the cameras 160can be a 360 camera that records 360-degree footage, also referred to asan omnidirectional camera. Examples of types of 360 cameras includedual-lens cameras; 360 cameras consisting of multiple cameras installedin a rig whose footage is stitched together into 360-degree footage;rotating cameras; and cameras which use 360 degree lenses. Examples of360 cameras include the GoPro™ Omni™, GoPro™ Odyssey™ Nokia™ OZO™,Facebook™ Surround360™, and Kodak™ Pixpro™. In accordance with anexample embodiment, once footage from the camera 160 is imported intothe system 101, it is no longer necessary to keep the camera 160connected to the system 101.

In accordance with an example embodiment, the processor 140 operatesunder stored program control and executes software modules 121 stored ina computer readable medium such as persistent memory, for example, in adata storage device 142. As illustrated in FIG. 2, in accordance with anexample embodiment, the software modules 121 include a 360-degree videopost-production application 172 and may also include operating systemsoftware 123, and other software applications 125.

In accordance with an example embodiment, software modules 121 or partsthereof may be temporarily loaded into volatile memory such as the RAM146 or the GPU's 141 volatile memory. The volatile memory is used forstoring runtime data variables and other types of data or information,as will be apparent to those skilled in the art. The GPU's 141 volatilememory is typically used for storing software modules 121 or partsthereof, runtime data variables and other types of data or informationrelated to graphics processing.

In accordance with an example embodiment, the system 101 stores data 127in an erasable persistent memory, which can include one or more datastorage devices 142. The data 127 can include data 127 related to the360-degree video post-production, including parts or all of the imagesor videos that are being edited or being used in post-production.

Although specific functions are described for various types of memory,these are only examples, and those skilled in the art will appreciatethat a different assignment of functions to types of memory could alsobe used.

In accordance with an example embodiment, the system 101 receives powerfrom a power supply 136, which typically receives power from an externalsource such as a plug into a power grid or an internal source, such as abattery.

Having provided an overview of an example system 101 on which exampleembodiments may be applied, specific features of example embodiments ofthe 360-degree video post-production methods and systems will now beexplained in greater detail.

Referring to FIG. 2, in accordance with an example embodiment a user cancreate a project 200 to start the 360-degree video post-productionprocess 202. As shown in FIG. 4, in accordance with an exampleembodiment, the 360-degree video post-production application 172 mayprovide the user with a user interface 400. The user can interact withthe user interface 400 using one or more of the input devices 106. Inaccordance with an example embodiment, the user interface 400 mayinclude a Project tab or page 402. In accordance with an exampleembodiment, the Project tab or page may have a Create Canvas 360 Projectsection 403. In accordance with an example embodiment, the user canprovide a name for the project in the project name field 404, andspecify the frame rate in the FPS field 410 and the duration or lengthof the project in the duration field 412.

In accordance with an example embodiment, the 360-degree composition ofthe post-production project is saved as a cube map 1000. In accordancewith an example embodiment, the 360-degree composition is saved by the360-degree video post-production application 172 as a cube map 1000 inthe project data 127 file. While an example embodiment may be describedin relation to a cube map 1000, a person skilled in the art would befamiliar with how images are transformed between different map types andwould be familiar with the changes necessary to accommodate other maptypes in the example embodiments.

Furthermore, while reference is generally made to 360-degree videos inexample embodiments, a person skilled in the art would understand thatexample embodiments can also be used for post-production of 360-degreepictures or stills. In accordance with an example embodiment,post-production can be applied to a 360-degree picture or still as if itwere a single frame of a 360-degree video.

In accordance with an example embodiment, the user can create a project200 based on an existing video composition from a video post-productiontool. In accordance with an example embodiment, a composition is aframework for a movie with its own timeline. In accordance with anexample embodiment, layers are elements that make up a composition, andcompositions can contain as many layers as necessary. Compositions cancontain as few as a single layer while others can include thousands oflayers. In accordance with an example embodiment, a composition caninclude multiple layers that represent components such as video andaudio footage items, animated text and vector graphics, still images,and lights. In accordance with an example embodiment, there are severalkinds of layers, including video and audio layers that are based onfootage items that you import, such as still images, movies, and audiotracks; layers that perform special functions, such as cameras, lights,adjustment layers and null objects; solid-color layers that are based onsolid-color footage items; synthetic layers that hold visual elementssuch as shape layers and text layers; and precomposition layers whichuse compositions as their source footage items. In accordance with anexample embodiment, a footage item can be added to a composition bycreating a layer for which the footage item is the source. In accordancewith an example embodiment, when you modify a layer, you do not affectits source footage item. In accordance with an example embodiment, youcan use the same footage item as the source for more than one layer anduse the footage differently in different layers. In accordance with anexample embodiment, layers can be arranged within a composition in spaceand time, and composite using transparency features to determine whichparts of underlying layers show through the layers stacked on top ofthem. In accordance with an example embodiment, the 360-degree videopost-production application 172 includes video composition tools, whichare known in the art, which the user can use to create a composition. Inaccordance with an example embodiment, the 360-degree videopost-production application 172 interacts with a video compositionapplication 125, which the user can use to create a composition.

In accordance with an example embodiment when the user creates a project200 based on an existing composition, the user can input or select acomposition in the Composition field 406 as well as input or select thefront camera for the composition in the Front Camera field 408. Inaccordance with an example embodiment, the project name can be enteredby the user into the project name field 404 or it can be automaticallyinput by the 360-degree video post-production application 172 based onthe file name for the composition. In accordance with an exampleembodiment, the frame rate can be automatically input into the FPS field410 and the length of the project can be automatically input into theduration field 412 based on the frame rate and length of thecomposition. In accordance with an example embodiment, the projectcreated based on an existing composition will have the same resolutionas the composition.

In accordance with an example embodiment, when a user creates a project200 based on an existing composition, 360-degree video footage createdbased on the composition will exist as a first video layer in theproject's 360-degree composition. In accordance with an exampleembodiment, when the first video layer is created from the existingcomposition, a cube map 1000 is built out of the existing composition.Methods for creating a cube map 1000 out of an existing composition arefurther described below. In accordance with an example embodiment, the360-degree video post-production application 172 uses the processor 140,the GPU 141 or a combination of the processor 140 and GPU 141 togenerate the 360-degree video footage based on the composition. Inaccordance with an example embodiment, the 360-degree videopost-production application 172 creates a new project data 127 file witha 360-degree composition with the 360-degree video footage based on theexisting composition as a first video layer in the project data 127file. In accordance with an example embodiment, the project is createdwhen the user presses the create button 414 after selecting acomposition in the Composition field 406.

In accordance with an example embodiment, when a user creates a project200 that is not based on an existing composition, the new projectincludes a new blank 360-degree composition with the requested framerate and duration and a blank cube map 1000. In accordance with anexample embodiment, a user can select “New Composition” in thecomposition field 406 to create a new blank project. In accordance withan example embodiment, the 360-degree video post-production application172 saves the 360-degree composition in the project data 127 file. Inaccordance with an example embodiment, the blank 360-degree compositionis created with a default resolution. In accordance with an exampleembodiment, the default resolution is 1920×1080. In accordance with anexample embodiment, the user can select a resolution for the new blankproject. In accordance with an example embodiment, the user can changethe resolution of the project after it is created. In accordance with anexample embodiment, the new blank project is created when the userpresses the create button 414 on a user interface 400.

In accordance with an example embodiment, a project can be deleted bypressing the Delete button 416. In accordance with an exampleembodiment, when a project is deleted, the 360-degree videopost-production application 172 deletes the project data 127 file.

In accordance with an example embodiment, the user can add additionallayers to the project's 360-degree composition throughoutpost-production. In accordance with an example embodiment, the data 127relating to the project layers are saved in the project data 127 file.

In accordance with an example embodiment, when a user creates a newproject 200, that project will have a default POV. The default POV canbe understood as the default, front view 302 of the project. Inaccordance with an example embodiment, when a user creates a new project200 based on an existing composition, the view 302 which was selected inthe Front Camera field 408 will be aligned with the default POV. Inaccordance with an example embodiment, when a user creates a new project200 that has a blank cube map 1000, the default POV will be the centerof the scene.

In accordance with an example embodiment, the user is shown theproject's 360-degree composition after the project is created. Inaccordance with an example embodiment, the user is shown the project's360-degree composition from a particular orientation in the omnidirectional camera viewer based on the default POV. In accordance withan example embodiment, the 360-degree video post-production application172 may show the omni directional camera viewer on one or more of thedisplays 104.

In accordance with an example embodiment, the omni directional cameraview shows a rectilinear view 302 of the project's 360-degreecomposition on one or more of the displays 104 based on a frame of aparticular POV. As shown in FIG. 3, a view 302 can be considered as asubset of the 360-degree composition based on a particular focal length,angle of view and orientation. Any orientation of the 360-degree spherecan be selected for a view 302, at any azimuth angle and elevationangle, for example. The angle of view, which may also be referred to asa field of view, describes the angular extent or size of the view 302.The focal length determines the extent of zoom of the view 302. Inaccordance with an example embodiment, a user can be considered asviewing a 360-degree scene 306 from the centre 304 of that scene 306.The particular focal length, angle of view and orientation for the frameof the POV the user has selected determines which view 302 of the360-degree scene 306 is shown to the user.

In accordance with an example embodiment, the 360-degree videopost-production application 172 uses the processor 140, the GPU 141 or acombination of the processor 140 and GPU 141 to generate a rectilinearversion of the view 302 based on the layers of the 360-degreecomposition in the project data 127 file. In accordance with an exampleembodiment, the resulting rectilinear version of the view can be storedtemporarily in volatile memory such as the RAM 146 or the GPU's 141volatile memory and output to one or more of the displays 104. Inaccordance with an example embodiment, the 360-degree videopost-production application 172 may save the resulting rectilinearversion of the view in the project data 127 file.

In accordance with an example embodiment, the user can use one or moreof the input devices 106 to change the omni directional camera viewer toshow different frames of the POV or to change the particular focallength, angle of view, or orientation of the view 302 in order todisplay other parts of the 360-degree scene 306. In accordance with anexample embodiment, the user can select which of the layers of theproject's 360-degree composition are displayed.

As shown in FIG. 11, in accordance with an example embodiment, the360-degree video post-production application 172 can provide the userwith an omni directional camera viewer interface 1100 on one or more ofthe displays 104. In accordance with an example embodiment, the omnidirectional camera viewer interface can include a viewing area 1102 todisplay a frame of the POV to the user and a list of layers 1104 in theproject, which the user can use to turn layers on or off using an inputdevice 106. In accordance with an example embodiment, the user canselect which frame of the project is being shown in the viewing area1102 by selecting a particular frame from the frame selector 1105 usingan input device 106.

As shown in FIG. 2, in accordance with an example embodiment, in orderto facilitate video-post production, the user can add POVs 217 to theproject. By defining the particular focal length, angle of view, andorientation for each frame of the project's 360-degree composition, aPOV identifies a particular view 302 of the 360 degree scene 306 foreach frame of the project.

In accordance with an example embodiment, the user can create a new POVfrom the omni directional camera viewer by modifying and animating theomni directional camera viewer's parameters. In accordance with anexample embodiment, the user can use one or more input devices 106 toadjust the viewing area 1002 of the omni directional camera viewerinterface 1100 and modify the focal length, angle of view, andorientation parameters to select a view 302 of the 360-degree scene 306.In accordance with an example embodiment, the user can select a view 302of the 360-degree scene 306 for a number of key frames from the360-degree composition, and the focal length, angle of view, andorientation for each frame in between the key frames is interpolated. Inaccordance with an example embodiment, the 360-degree videopost-production application 172 uses the processor 140, the GPU 141 or acombination of the processor 140 and GPU 141 to interpolate the focallength, angle of view, and orientation for each frame between the keyframes. In accordance with an example embodiment, the user may selectone focal length, angle of view, and orientation for the entire360-degree composition as a POV. In accordance with an exampleembodiment, the user may manually enter or modify the focal length,angle of view, and orientation for the POV.

As shown in FIG. 7, in accordance with an example embodiment, the360-degree video post-production application 172 will provide the userwith a user interface 400 which includes a POV tab or page 702. Aftercreating a new POV, the user can enter a name for the POV into the Namefield 704 and add a POV 217 to the project by using the save button 706.In accordance with an example embodiment, when a user adds a POV 217 tothe project, the 360-degree video post-production application 172 cansave the information for the POV in the project data 127 file.

In accordance with an example embodiment, the user interface 400 mayprovide the user with a list of the POVs 708. The user can select a POVfrom the list of POVs 708. In accordance with an example embodiment, theuser can select a specific POV from the list of POVs 708. In accordancewith an example embodiment, the user can open a specific POV by doubleclicking on the POV or pressing the load button 710. In accordance withan example embodiment, the POV can be viewed using the omni directionalcamera viewer. In accordance with an example embodiment, when a POV isopened, the focal length, angle of view, and orientation for the omnidirectional camera viewer are matched to the focal length, angle ofview, and orientation of the opened POV, and the user is shown aparticular frame of the opened POV. In accordance with an exampleembodiment, the frame of the POV shown to the user in the omnidirectional camera viewer may be the first frame, or it may be the frameof the 360-degree composition that was being viewed by the userimmediately before opening the POV. In accordance with an exampleembodiment, the omni directional camera viewer allows the user to lookaround the 360-degree composition after loading the POV. In accordancewith an example embodiment, the POV is shown in the omni directionalcamera viewer interface 1100.

In accordance with an example embodiment, the 360-degree videopost-production application 172 can export a POV to the system's 101data storage devices 142, an external hard drive connected through theI/O subsystem 150 or serial port 152, other persistent memory connectedto the system 101, or an external communication link or interface. Inaccordance with an example embodiment, the selected POV can be exportedusing the export button 712 and then providing the 360-degree videopost-production application 172 with an export location for the POVinformation. In accordance with an example embodiment, the POVinformation includes the parameters for the POV.

In accordance with an example embodiment, exporting may be helpful whena 360-degree video is being edited in different instances of the videopost-production application 172 or when different project data 127 filesare used with respect to the same 360-degree video. In accordance withan example embodiment, a POV may also be exported in order to save acopy for later use.

In accordance with an example embodiment, a POV can be deleted from theproject. In accordance with an example embodiment, a POV can be deletedby selecting the POV from the list of POVs 708 and then using the deletebutton 714. In accordance with an example embodiment, when a POV isdeleted, the 360-degree video post-production application 172 deletesthe information related to that POV from the project data 127 file.

In accordance with an example embodiment, a user can render a POV 219 inorder to create a rectilinear video based on the POV. In accordance withan example embodiment, to render a rectilinear video based on the POV,the 360-degree video post-production application 172 creates a renderedvideo where each frame of the rendered video is the rendered view 302 ofthe corresponding frame of the 360-degree composition of the project, asdefined by the POV. In accordance with an example embodiment, the layersfor the frame of the 360-degree composition are flattened into a singlelayer when rendering the 360-degree scene.

In accordance with an example embodiment, a user can render a POV 219 byopening the POV, and then selecting a render POV option (not shown) inthe unified omni directional camera viewer interface 1100. In accordancewith an example embodiment, the 360-degree video post-productionapplication 172 can save the rendered video in the project data 127file. In accordance with an example embodiment, the 360-degree videopost-production application 172 can save the rendered video as aseparate file in the system's 101 data storage devices 142, on anexternal hard drive connected through the I/O subsystem 150 or serialport 152, in a camera's 160 internal memory, in other persistent memoryconnected to the system 101, or the rendered video may be sent throughan external communication link or interface.

As shown in FIG. 2, in accordance with an example embodiment, the usercan add a 360 pass 215 into the project. In accordance with an exampleembodiment, a 360 pass can include video or still image files from a360-degree camera, computer generated 360-degree footage or images,and/or 360-degree videos or images created by stitching together footagefrom a number of cameras.

Examples of video files include but are not limited to Animated GIF(GIF); DV (in MOV or AVI container, or as containerless DV stream);ElectricImage (IMG, EI); Filmstrip (FLM); Flash (SWF); MPEG formats(MPEG, MPE, MPG, M2V, MPA, MP2, M2A, MPV, M2P, M2T, VOB, MOD, AC3, MP4,M4V, M4A); Open Media Framework (OMF); QuickTime™ (MOV); Adobe™Photoshop™ with video layer (PSD); Video for Windows™ (AVI, WAV); andWindows™ Media File (WMV, WMA, ASF). Examples of images files includebut are not limited to Adobe™ Illustrator™ (AI, AI4, AI5, EPS, PS);Adobe™ PDF (PDF); Adobe™ Photoshop™ (PSD); Bitmap (BMP, RLE, DIB);Camera raw (TIF, CRW, NEF, RAF, ORF, MRW, DCR, MOS, RAW, PEF, SRF, DNG,X3F, CR2, ERF); Cineon™ (CIN, DPX;); Discreet™ RLA/RPF (RLA, RPF); EPS;JPEG (JPG, JPE); Maya™ camera data (MA); Maya™ IFF (IFF, TDI; 16 bpc);OpenEXR (EXR; 32 bpc); PBM (8, 16, and 32 bpc); PCX; PICT (PCT); Pixar™(PXR); Portable Network Graphics (PNG); Radiance™ (HDR, RGBE, XYZE); SGI(SGI, BW, RGB); Softimage™ (PIC); Targa™ (TGA, VDA, ICB, VST); TIFF(TIF). 360-degree video files can be saved using different map types,including equirectangular map 900 or cube map 1000.

As shown in FIG. 4, in accordance with an example embodiment, the360-degree video post-production application 172 provides the user witha user interface 400 which includes a 360 tab or page 502. In accordancewith an example embodiment, the user can input or select the filecontaining the 360 media that the user wants to add to the project inthe 360 Media field 504. In accordance with an example embodiment, thefile containing the 360 media may have been saved on the system's 101data storage devices 142, on an external hard drive connected throughthe I/O subsystem 150 or serial port 152, in a camera's 160 internalmemory, in other persistent memory connected to the system 101, or thefile may come from an external communication link or interface.

In accordance with an example embodiment, the user identifies the maptype of the 360 pass in the 360 Map Type field 506. In accordance withan example embodiment, the map type is identified automatically from thefile containing the 360 media.

In accordance with an example embodiment, the user can add a name toidentify the particular pass in the Name field 510.

In accordance with an example embodiment, when the user directs the360-degree video post-production application 172 to add the 360 pass215, for example by pressing the Add button 512, the 360-degree videopost-production application 172 begins importing the 360-degree passinto the project data 127 file.

In accordance with an example embodiment, a cube map 1000 is built outof the 360 media. In accordance with an example embodiment, the cubemap's 1000 orientation and rotation is set so that the front face of the360 media is aligned with the default POV. In accordance with an exampleembodiment, the cube map's 1000 orientation and rotation is set so thatthe front face of the 360 media is aligned with a project POV selectedby the user. In accordance with an example embodiment, the user selectsthe project POV that will be used using the Point of View field 508. Inaccordance with an example embodiment, the methods for creating a cubemap 1000 out of a 360-degree composition, further described below, areused to build a cube map 1000 out of the 360 media.

In accordance with an example embodiment, the cube map 1000 is added asa new video layer in the project's 360-degree composition. In accordancewith an example embodiment, where there are existing layers in theproject, the new 360 pass layer can be placed above, in between, orunder any of the existing layers in the project. In accordance with anexample embodiment, the new layer is saved in the project data 127 file.

In accordance with an example embodiment, the 360 media for the 360 passhas the same duration and length as the 360-degree composition of theproject. In accordance with an example embodiment, if the frame rate ofthe 360 media does not match the project frame rate, the user willreceive a confirmation window asking the user whether the user wants tocontinue with the incorrect frame rate or cancel the operation. Inaccordance with an example embodiment, an imported 360 pass which doesnot have the same duration as the 360-degree composition will bemodified to match the 360-degree composition's duration. In accordancewith an example embodiment, where the 360 pass is longer than the360-degree composition, the 360 pass is edited to remove part of thebeginning, part of the end, or part of the beginning and part of the endof the 360 pass. In accordance with an example embodiment, where the 360pass is longer than the 360-degree composition, the 360-degreecomposition is extended at the beginning, the end, or the beginning andend of the 360-degree composition in order to lengthen the duration ofthe 360-degree composition, the extension consisting of either blankfootage or a still based on the first or last frame of the 360-degreecomposition. In accordance with an example embodiment, where the 360pass is shorter than the 360-degree composition, the 360 pass isextended at the beginning, the, end or the beginning and end of the passin order to lengthen the duration of the 360 pass, the extensionconsisting of either blank footage or a still based on the first or lastframe of the 360 pass. In accordance with an example embodiment, wherethe 360 pass is shorter than the 360-degree composition, the 360-degreecomposition is edited to remove part of the beginning, part of the end,or part of the beginning and part of the end of the 360-degreecomposition in order to shorten the duration of the 360-degreecomposition. In accordance with an example embodiment, the system 101uses the processor 140, the GPU 141 or a combination of the processor140 and GPU 141, to edit the length of the 360 pass or 360-degreecomposition.

In accordance with an example embodiment, adding a 360 pass 215 may bepart of the process to create a project 200. In accordance with anexample embodiment, the user selects a 360 pass to import as part of theprocess to create a new project 200. In accordance with an exampleembodiment, the frame rate and duration for the project may beautomatically detected based on information from the 360 pass file. Inaccordance with an example embodiment, the new project is created withthe 360 pass as the first video layer of the project's 360-degreecomposition. In accordance with an example embodiment, the 360-degreevideo post-production application 172 creates a new project data 127file, which includes a first video layer with the 360 pass. Inaccordance with an example embodiment, the front camera view 302 of the360 pass which was added when creating the new project 200 will bealigned with the default POV of the project.

As shown in FIG. 2, in accordance with an example embodiment, the usercan apply post-production editing by adding a picture-in-picture (PIP)216 to a particular POV. In accordance with an example embodiment,adding a PIP 216 allows the user to make edits in a particular POV, forexample, color corrections, addition of computer generated images, orcomposition of rectilinear videos into the POV. In accordance with anexample embodiment, the edits made in that POV are added as a new PIPlayer in the 360-degree composition of the project. In accordance withan example embodiment, the layers are saved as part of the project data127 file.

In accordance with example embodiments, the layers are saved to memory(e.g. data storage device 142).

In accordance with an example embodiment, PIP editing can be performeddirectly in the omni directional camera viewer in a method referred toas solid PIP. In solid PIP, a new PIP layer is added to the 360-degreecomposition of the project at the position of the particular POV. Inaccordance with an example embodiment, the new solid PIP layer is addedas the top layer of the 360-degree composition. In accordance with anexample embodiment, the solid PIP layer is a solid layer comprised of asingle solid color. In accordance with an example embodiment, the solidPIP layer is an adjustment layer, which is a flat transparent layer thatcan be used to add adjustments to any layers underneath the adjustmentlayer.

In accordance with an example embodiment, the PIP editing can beperformed on a sub-composition of the project from the selected POV. Inthis method, referred to as 360 PIP, a sub-composition of the project iscreated by flattening the 360-degree composition's layers from theselected POV into a single layer.

In accordance with an example embodiment, the sub-composition is savedin the project data 127 file. In accordance with an example embodiment,rectilinear post-production visual effects methods and tools can beapplied to the sub-composition since the selected POV defines arectilinear portion of the 360-degree composition.

In accordance with an example embodiment, the 360-degree videopost-production application 172 includes rectilinear post-productionvisual effects methods and tools that the user can use to apply visualeffects to the sub-composition. In accordance with an exampleembodiment, the 360-degree video post-production application 172 mayinteract with a rectilinear post-production application 125 by providingit with the sub-composition. Rectilinear post-production methods canthen be applied to the sub-composition using the rectilinearpost-production application 125. Edits to the sub-composition arecommunicated back to the 360-degree video post-production application172.

In accordance with an example embodiment, by overlaying thesub-composition layer with the same focal length, angle of view andorientation as the POV associated with the 360 PIP, the sub-compositionis integrated into the 360-degree composition of the project.

In accordance with an example embodiment rectilinear media files, whichcould be video or image files, can be overlaid over a POV in the360-degree video in a method referred to as media PIP. In accordancewith an example embodiment, the media file of the media PIP is alignedwith the same focal length, angle of view and orientation as the POV inorder to overlay the POV with the associated media file.

In accordance with an example embodiment, the imported media file for amedia PIP may be a video file rendered from a POV in the project. Inaccordance with an example embodiment, the rendered media file can beedited using different video post-production applications 125 in thesystem 101 or exported to and edited in a different videopost-production system. In accordance with an example embodiment, theuser would typically use the same POV from which the media file wasrendered when importing the rendered media file back into the projectusing Media PIP.

In accordance with an example embodiment, the media file contains avideo that has the same duration and length as the 360-degreecomposition of the project. In accordance with an example embodiment,the media file contains an image, in which case a duplicate of thatimage is created for each frame of the 360-degree composition. Inaccordance with an example embodiment, an imported media file thatcontains a video which does not have the same duration of the 360-degreecomposition will be modified to match the 360-degree composition'sduration. In accordance with an example embodiment, where the video inthe media file is longer than the 360-degree composition, the video inthe media file is edited to remove part of the beginning, part of theend, or part of the beginning and part of the end of the video in themedia file. In accordance with an example embodiment, where the video inthe media file is longer than the 360-degree composition, the 360-degreecomposition is extended at the beginning, the end, or the beginning andend of the 360-degree composition in order to lengthen the duration ofthe 360-degree composition, the extension consisting of either blankfootage or a still based on the first or last frame of the 360-degreecomposition. In accordance with an example embodiment, where the videoin the media file is shorter than the 360-degree composition, the videoin the media file is extended at the beginning, the, end or thebeginning and end of the pass in order to lengthen the duration of thevideo in the media file, the extension consisting of either blankfootage or a still based on the first or last frame of the video in themedia file. In accordance with an example embodiment, where the video inthe media file is shorter than the 360-degree composition, the360-degree composition is edited to remove part of the beginning, partof the end, or part of the beginning and part of the end of the360-degree composition in order to shorten the duration of the360-degree composition. In accordance with an example embodiment, thesystem 101 uses the processor 140, the GPU 141 or a combination of theprocessor 140 and GPU 141, to edit the length of the video in the mediafile or the length of the 360-degree composition.

As shown in FIG. 4, in accordance with an example embodiment, the360-degree video post-production application 172 will provide the userwith a user interface 400 which includes a PIP tab or page 602. Inaccordance with an example embodiment, the user can select a video fileto import using Media PIP in the Media field 604. In accordance with anexample embodiment, the user can select 360 PIP in the Media Field 604if the user wants to perform 360 PIP editing. In accordance with anexample embodiment, the user can select Solid PIP from the Media Field604 if the user wants to perform Solid PIP editing. In accordance withan example embodiment, the user can select the POV to which the PIPediting will be applied in the Point of View field 606. In accordancewith an example embodiment, the user can provide a name to identify thePIP edit in the Name field 608. In accordance with an exampleembodiment, a user can add a PIP 216 to the project by pressing the Addbutton 610, which creates a new PIP layer for the project based on theselected PIP editing type, as has been described. In accordance with anexample embodiment, the 360-degree video post-production application 172creates the new PIP layer and saves it in the project data 127 file.

In accordance with example embodiments, the PIP layer is saved to memory(e.g. data storage device 142).

In accordance with an example embodiment, the PIP tab or page 602includes a list of PIP layers 612. In accordance with an exampleembodiment, a user can select a PIP layer from the list of PIP layers612 and press the delete button 616 to delete the PIP layer. Inaccordance with an example embodiment, when a PIP layer is deleted, the360-degree video post-production application 172 deletes informationrelated to the PIP layer from the project data 127 file.

In accordance with an example embodiment, when a user wants to makefurther edits to a PIP, the user can select a PIP layer from the list ofPIP layers 612 and press the open button 614 to open the PIP for furtherediting. In accordance with an example embodiment, the corresponding PIPlayer is shown in the omni directional camera viewer for editing. Inaccordance with an example embodiment, when a PIP is opened, a card forthe selected PIP is created. In accordance with an example embodiment,when a PIP has a card saved in the project data 127 file, that card isused when the PIP is opened. In accordance with an example embodiment,this card can then be edited in the same manner as described for 360PIP.

As seen in FIG. 2, in accordance with an example embodiment, a user canadd tracking 218 information as a new layer or layers in the project.Referring to FIG. 13, an add tracking 218 method flowchart 1300 inaccordance with an example embodiment is shown. In accordance with anexample embodiment, a user applies tracking to a particular POV. Inaccordance with an example embodiment, a user can use that trackinginformation to assist in PIP editing, where it is desired to add visualeffects that follow or are affected by a tracked object. In accordancewith an example embodiment, the tracking information for a particularPOV can be transformed into tracking information for the 360-degreevideo of the project. In accordance with an example embodiment, trackinginformation can be used to solve the camera for a particular POV, as isknown by the person skilled in the art. In accordance with an exampleembodiment, a solved camera for a particular POV can be used to solvethe camera for the 360-degree video of the project.

As shown in FIG. 8, in accordance with an example embodiment, the360-degree video post-production application 172 will provide the userwith a user interface 400 which includes a Tracking tab or page 802. Inaccordance with an example embodiment, the user can select the 360 passin the 360 Pass field 804 and the POV in the Point of View field 806 onwhich the tracking will be performed 1302. The selected 360 passdetermines which 360 pass is used as the source for the 360-degree scene306 when generating the POV's view 302.

In accordance with an example embodiment, the 360-degree videopost-production application 172 tracks the 360 pass from the selectedPOV 1304. In accordance with an example embodiment, the 360-degree videopost-production application 172 tracks the 360 pass from the selectedPOV 1304 when the user presses the Track Camera button 808. Inaccordance with an example embodiment, by applying tracking to aparticular POV, rectilinear video tracking methods, tools and algorithmscan be used. In accordance with an example embodiment, tracking isperformed by the 360-degree video post-production application 172 byapplying rectilinear tracking algorithms using the processor 140, theGPU 141 or a combination of the processor 140 and GPU 141. In accordancewith an example embodiment, the 360-degree video post-productionapplication 172 interacts with a video tracking application 125 whichgenerates tracking data for the POV. In accordance with an exampleembodiment, the tracking data for the POV is saved to the project data127 file.

In accordance with an example embodiment, tracking is performed byexisting video tracking applications 125 or video tracking systems on avideo file rendered from a POV in the project. In accordance with anexample embodiment, tracking information is recorded when a video isrecorded or generated. In accordance with an example embodiment,tracking information generated in other tracking applications 125, inother video tracking systems, or when a video is recorded can beimported as tracking information for a particular POV of the project.

In accordance with an example embodiment, the tracking informationconsists of point cloud layers and a tracked camera layer, which is avirtual camera whose parameters are calculated by solving the camera forthe tracked POV. In accordance with an example embodiment, the virtualcamera parameters are position, focal length, angle of view andorientation. In accordance with an example embodiment, the point cloudconsists of a series of layers that hold a 3D position relative to thecamera.

In accordance with an example embodiment, the video post-productionapplication 172 will display the POV along with track points from thepoint cloud on one or more of the displays 104 and the user will be ableto edit or change the track points using one or more of the inputdevices 106.

In accordance with an example embodiment, the user can select one ormore layers from the point cloud along with the tracked camera layerfrom a tracked POV to be imported into the project's 360-degreecomposition. In accordance with an example embodiment, all layers fromthe point cloud along with the tracked camera layer from a tracked POVare imported into the project's 360-degree composition.

In accordance with an example embodiment, the point cloud layers arelinked to the tracked camera layer 1306 before the tracking informationis imported into the project's 360-degree composition as a new layer.When point cloud layers are linked to a tracked camera layer 1306, anysubsequent changes made to the camera's position or orientation,including any scaling of the camera's position, are also applied to thepoint cloud, meaning the relative position of the point cloud withrespect to the tracked camera remains consistent when the point cloud islinked to the tracked camera.

In accordance with an example embodiment, when the tracking informationis imported into the project's 360-degree composition as new layers, thetracked camera's first frame is aligned with the first frame of thetracked POV 1308 in the project's 360-degree composition. In accordancewith an example embodiment, since the point cloud layers are linked tothe tracked camera layer, aligning the first frame of the tracked camerawith the first frame of the tracked POV 1308 also aligns the point cloudin the project's 360-degree composition.

In accordance with example embodiments, once the first frame of thetracked camera is aligned in the 360-degree composition with the firstframe of the tracked POV 1308, the point cloud layers and the trackedcamera layer are saved in the project data 127 file. In accordance withexample embodiments, once the first frame of the tracked camera isaligned in the 360-degree composition with the first frame of thetracked POV 1308, the point cloud layers and the tracked camera layerare saved to memory (e.g. data storage device 142).

In accordance with an example embodiment, the tracked camera layer isautomatically used to stabilize the project's 360-degree compositionwhen tracking information is first imported into the project's 360degree composition.

In accordance with an example embodiment, if it is the first timetracking information is being imported into the project's 360-degreecomposition, the position of the tracked camera is copied to all otherexisting POVs in the project 1309, since they are all based on footagefrom the same camera.

In accordance with an example embodiment, if it is the first timetracking information is being imported into the project's 360-degreecomposition, the orientation of all the existing POVs are updated basedon where each POV is facing relative to the tracked POV. In accordancewith an example embodiment, the orientation differences between thetracked POV and the other POVs are calculated 1310. In accordance withan example embodiment, after the orientation differences are calculated1310 and the tracked camera layer is aligned with the first frame of thetracked POV 1308, the orientation information from the now alignedtracked camera is copied over to the tracked POV 1312. In accordancewith an example embodiment, the orientations of the other POVs areupdated by adding the difference calculated to the orientationinformation of the now aligned tracked camera 1314. For example, if aPOV is facing the exact opposite direction of the tracked POV, then itsorientation will be the reverse of the tracked POV.

In accordance with an example embodiment, the orientation of the cubemap 1000 of all existing 360 passes in the 360-degree composition isthen aligned with the inversion of the orientation of the POV associatedwith the 360 pass 1316. By aligning the orientation of the cube maps1000 with the inversion of the orientation of their associated POV 1316,the movement of the image in the 360 media is canceled out. Since theorientation of all existing POVs were updated based on the trackedcamera, the POVs will still display the same view of the 360 compositionafter the 360-degree composition is stabilized. In accordance with anexample embodiment, a new stabilized POV with all orientation values setto zero is created 1318.

In accordance with an example embodiment, when subsequent trackinginformation is imported into the 360-degree composition, after thetracked layer is aligned with the first frame of the tracked POV, thecamera's position parameters are scaled down or up to approximatelymatch the scale of the originally added tracking information 1320. Inaccordance with an example embodiment, the scale used is estimated orapproximated and therefore needs to be scaled so that distances matchthe original tracking information. In accordance with an exampleembodiment, the furthest point from the first camera position isidentified, and the camera path of the newly imported tracked cameralayer is then scaled until the difference between the first cameraposition and the furthest camera position of the newly imported trackedcamera layer matches the difference in the original tracking camerapositions. Once the scene is scaled, the camera paths shouldapproximately align if the tracking data is accurate since they arebased on the same camera footage. By aligning the camera paths, thenewly added point clouds, which are linked to the tracked camera andtherefore scaled along with the tracked camera, should approximatelyline up in the 360-degree composition with the tracked POV, the 360passes and other point clouds in the 360-degree composition.

In accordance with an example embodiment, the point cloud layers areunlinked from the tracked camera layer 1322 after the trackinginformation is imported into the project's 360-degree composition.

In accordance with an example embodiment, the 360-degree videopost-production application 172 uses the processor 140, the GPU 141 or acombination of the processor 140 and GPU 141 to import the POV'stracking information into the project's 360-degree composition Inaccordance with an example embodiment, the tracking information is savedto the project data 127 file.

As shown in FIG. 8, in accordance with an example embodiment, theTracking tab or page 802 includes a Select Layers field 810 to selectthe tracking layers and tracked camera layer to import. In accordancewith an example embodiment, the user can select the POV which thetracking layers and tracked camera layer relate to in the Point of Viewfield 812. In accordance with an example embodiment, the trackinginformation from the tracking layers and tracked camera layer istransformed into tracking information for the 360-degree compositionwhen the user presses the Add Tracking Data button 814.

In accordance with an example embodiment, once tracking information forthe 360-degree composition of the project is imported, the trackinginformation can be used in the same manner that traditional rectilineartracking data is used. In accordance with an example embodiment, thetracking information is available in POVs in which the point cloud isvisible. In accordance with an example embodiment, the point cloud dataremains accurate as the user looks around the 360-degree compositionfrom different views.

As shown in FIG. 2, in accordance with an example embodiment, a user cangenerate a new 360-degree video 214 from the project. In accordance withan example embodiment, the front of the newly generated 360-degree videocan be aligned with any POV in the project. In accordance with anexample embodiment, the newly generated 360-degree video can begenerated using different map types, for example an equirectangular map900 or a cube map 1000. In accordance with an example embodiment, thenew generated 360-degree video can be generated using differentresolutions, for example 8000×4000, 4000×2000, or 2000×1000 forequirectangular map-types; 8000×6000, 4000×3000, 2000×1500 for anH-cross cube map; or 6000×8000, 3000×4000, 1500×2000 for a V-cross cubemap.

As shown in FIG. 4, in accordance with an example embodiment, theProject tab or page 402 may include an Output in 360 section 417. Inaccordance with an example embodiment, the user can select the POV to beused as the front camera for the newly generated 360-degree video fromthe Point of View field 418; select the map type to be used in the newlygenerated 360-degree video in the 360 Map Type field 420; and select theresolution for the newly generated 360-degree video in the Size field422. In accordance with an example embodiment, the user can press theUpdate button 424 to generate the newly 360-degree video 214 using theselected POV, map type and resolution. In accordance with an exampleembodiment, the user can press the Open button 426 to open an existing360-degree video after it has been closed by the user. In accordancewith an example embodiment, the user can press the Delete button 428 todelete a 360-degree video. In accordance with an example embodiment,deleting a 360-degree video can be useful to reduce the size of theproject data 127 file if the 360-degree video is saved in the projectdata 127 file.

In accordance with example embodiments, the 360-degree compositiontogether with one or more layers can be viewed by the user on a display.In accordance with example embodiments, the user can select which of thelayers to include when viewing the 360-degree composition. In accordancewith example embodiments, the 360-degree composition together with theone or more layers are output to a display.

In accordance with example embodiments, the display is a computermonitor. In accordance with example embodiments, a rectilinear subsetview of the 360-degree composition together with the one or more layersis output to the computer monitor and the user can change the view beingdisplayed using one of the input devices 106.

In accordance with example embodiments, the display is a 360-degreedisplay. As shown in FIG. 14, in accordance with example embodiments,the 360-degree display is a head-mounted display 1400. In accordancewith example embodiments, the head-mounted display 1400 has a cover1402, padding 1404 and a strap 1406. The user can put on thehead-mounted display 1400 by placing the padding 1404 around the user'seyes and holding it in place with the strap 1406. In accordance withexample embodiments, the head-mounted display 1400 has a screen andsensors within the cover 1402. The screen displays a portion of the360-degree composition together with the one or more layers based on theposition of the user's head.

In accordance with an example embodiment, when a user decides togenerate a new 360-degree video 214 from the project, a new cube map1000 of the 360-degree video is generated based on the POV selected bythe user to be used as the front camera.

In accordance with an example embodiment, 6 identical copies of theproject's 360-degree composition are created, one corresponding to eachface of the cube map 1000 for the new 360-degree video. In accordancewith an example embodiment, the orientation of the faces of the 6compositions correspond to the orientation of the POV, 90 degrees upfrom the selected POV, 90 degrees down from the selected POV, 90 degreeleft from the selected POV and 90 degrees right from the selected POV,and behind the selected POV. In accordance with an example embodiment,each composition is aligned with its corresponding face and then resizedto a single cube face's size based on the selected resolution. Inaccordance with an example embodiment, each composition is resized tothe selected resolution divided by three. In accordance with an exampleembodiment, the 6 compositions are then placed in a new cube map 1000.In accordance with an example embodiment, where the desired output isother than a cube map 1000, the cube map 1000 is then converted into thedesired format.

In accordance with an example embodiment, the 360-degree videopost-production application 172 uses the processor 140, the GPU 141 or acombination of the processor 140 and GPU 141 to generate a new360-degree video 214 from the project. In accordance with an exampleembodiment, the 360-degree video post-production application 172 cansave the newly generated 360-degree video in the project data 127 file.In accordance with an example embodiment, the 360-degree videopost-production application 172 can save the newly generated 360-degreevideo as a separate file in the system's 101 data storage devices 142,on an external hard drive connected through the I/O subsystem 150 orserial port 152, in a camera's 160 internal memory, in other persistentmemory connected to the system 101, or the new 360-degree video may besent through an external communication link or interface.

In accordance with an example embodiment, a user may generate a new360-degree video 214 using a stabilized POV. The new generated360-degree video will then also be stabilized.

As shown in FIG. 2, in accordance with an example embodiment, aftercreating a new project 200, a user can decide to do any type ofpost-production editing on the project, including adding a 360 pass 215,adding a PIP 216, adding a POV 217, adding tracking 218, generating anew 360 degree video 214 or rendering a POV 219. In accordance with anexample embodiment, the user can decide to apply further post-productionediting of any type. In accordance with an example embodiment, the usercan perform more than one type of post-production editing or more thanone instance of a particular type of post-production editing.

In accordance with an example embodiment, a user may add multiple POVs217 and then render those POVs 218. Those rendered POVs can then beprovided to other users who will edit the rendered POVs in differentvideo post-production applications 125 or other video post-productionssystems. Once the users have completed editing the rendered POVs, theedited rendered POVs can be added back into the project as Media PIPs byadding PIPs 216.

In accordance with an example embodiment, aspects of the 360-degreevideo post production may be implemented on one or more applications 125which interact with the 360-degree video post-production application172. In accordance with an example embodiment, the 360-degree videopost-production application 172 is a software plug-in providing360-degree video post-production features to a video post-productionapplication 125.

While some of the present embodiments are described in terms of methods,a person of ordinary skill in the art will understand that presentembodiments are also directed to various apparatus such as processors,servers and communication networks including components for performingat least some of the aspects and features of the described methods, beit by way of hardware components, software or any combination of thetwo, or in any other manner. Moreover, an article of manufacture for usewith the apparatus, such as a pre-recorded storage device or othersimilar non-transitory computer readable medium including programinstructions recorded thereon, or a computer data signal carryingcomputer readable program instructions may direct an apparatus tofacilitate the practice of the described methods. It is understood thatsuch apparatus, articles of manufacture, and computer data signals alsocome within the scope of the present example embodiments.

In the Figures, as applicable, at least some or all of the illustratedsubsystems or blocks may include or be controlled by a processor, whichexecutes instructions stored in a memory or computer readable medium.

The term “computer readable medium” as used herein includes any mediumwhich can store instructions, program steps, or the like, for use by orexecution by a computer or other computing device including, but notlimited to: magnetic media, such as a diskette, a disk drive, a magneticdrum, a magneto-optical disk, a magnetic tape, a magnetic core memory,or the like; electronic storage, such as a random access memory (RAM) ofany type including static RAM, dynamic RAM, synchronous dynamic RAM(SDRAM), a read-only memory (ROM), a programmable-read-only memory ofany type including PROM, EPROM, EEPROM, FLASH, EAROM, a so-called “solidstate disk”, other electronic storage of any type including acharge-coupled device (CCD), or magnetic bubble memory, a portableelectronic data-carrying card of any type including COMPACT FLASH,SECURE DIGITAL (SD-CARD), MEMORY STICK, and the like; and optical mediasuch as a Compact Disc (CD), Digital Versatile Disc (DVD) or BLU-RAYDisc.

Variations may be made to some example embodiments, which may includecombinations and sub-combinations of any of the above. The variousembodiments presented above are merely examples and are in no way meantto limit the scope of this disclosure. Variations of the innovationsdescribed herein will be apparent to persons of ordinary skill in theart having the benefit of the example embodiments, such variations beingwithin the intended scope of the present disclosure. In particular,features from one or more of the above-described embodiments may beselected to create alternative embodiments comprised of asub-combination of features, which may not be explicitly describedabove. In addition, features from one or more of the above-describedembodiments may be selected and combined to create alternativeembodiments comprised of a combination of features which may not beexplicitly described above. Features suitable for such combinations andsub-combinations would be readily apparent to persons skilled in the artupon review of the present disclosure as a whole. The subject matterdescribed herein intends to cover and embrace all suitable changes intechnology.

Certain adaptations and modifications of the described embodiments canbe made. Therefore, the above discussed embodiments are considered to beillustrative and not restrictive.

What is claimed is:
 1. A method for 360-degree video post-production,the method being performed by at least one processor, the methodcomprising: displaying an interface screen for a 360-degree compositionon an output interface device comprising a display, the interface screenconfigured to display an omni directional camera viewer that isconfigured to display a rectilinear view of the 360-degree compositionas a consequence to orientation navigating through the omni directionalcamera viewer; identifying a point of view comprising receivingselection through the interface screen of a rectilinear view displayedin the interface screen; applying video post-production editing to theidentified point of view, comprising tracking the identified point ofview to identify tracking information comprising one or more point cloudlayers and a tracked camera layer; linking the one or more point cloudlayers to the tracked camera layer; aligning the one or more point cloudlayers in the 360-degree composition with the identified point of view;and saving one or more layers, comprising the point cloud layers and thetracked camera layer, to memory.
 2. A method for 360-degree videopost-production in accordance with claim 1, further comprising:identifying an existing or additional point of view through theinterface screen; said applying video post-production editing comprisingvideo post-production editing of the identified existing or additionalpoint of view to create one or more additional layers; aligning the oneor more additional layers in the 360-degree composition with theidentified existing or additional point of view; and saving the one ormore additional layers to memory.
 3. A method for 360-degree videopost-production in accordance with claim 2, wherein: said applying videopost-production editing comprises importing media of a media file.
 4. Amethod for 360-degree video post-production in accordance with claim 3,wherein the media file is a 360-degree media file of 360-degree footagetaken from a 360-degree camera.
 5. A method for 360-degree videopost-production in accordance with claim 1, further comprisingstabilizing the 360-degree composition based on the trackinginformation.
 6. A method for 360-degree video post-production inaccordance with claim 5, wherein said stabilizing further comprises:orienting the 360-degree composition with an inversion of an orientationof the tracked camera layer; and updating an orientation of theidentified point of view to match the orientation of the tracked cameralayer after the tracked camera layer is aligned with the identifiedpoint of view.
 7. A method for 360-degree video post-production inaccordance with claim 6, wherein said stabilizing further comprises:selecting one or more points of view in addition to the identified pointof view; calculating a difference in orientation between the selectedone or more points of view and the identified point of view before theorientation of the identified point of view is updated to match theorientation of the tracked camera layer; and updating the orientation ofthe one or more selected points of view to match the addition of thecalculated difference and the orientation of the tracked camera layerafter the tracked camera layer is aligned with the identified point ofview.
 8. A method for 360-degree video post-production in accordancewith claim 1, wherein said orientation navigating and/or said receivingselection are controlled using an input interface device.
 9. A methodfor 360-degree video post-production in accordance with claim 1, whereinthe displayed rectilinear view is also the consequence of navigating oneor more of the angle of view and focal length.
 10. A method for360-degree video post-production in accordance with claim 1, wherein thedisplayed rectilinear view is generated by identifying and displaying asubset of the 360-degree composition corresponding to the displayedrectilinear view.
 11. A method for 360-degree video post-production inaccordance with claim 1, wherein the identified point of view includes aparticular focal length, angle of view, and orientation of the360-degree composition.
 12. A method for 360-degree videopost-production in accordance with claim 1, wherein the identified pointof view is selectable from any viewing angle.
 13. A method for360-degree video post-production in accordance with claim 1, furthercomprising rendering a new 360-degree video file which renders the360-degree composition together with the one or more layers, and savingthe new 360-degree video file to the memory.
 14. A method for 360-degreevideo post-production in accordance with claim 13, wherein the new360-degree video file is rendered by flattening all of the one or morelayers in the 360-degree composition.
 15. A method for 360-degree videopost-production in accordance with claim 2, wherein the videopost-production editing comprises: generating a sub-composition of theidentified existing or additional point of view; and said applying videopost-production editing comprises applying rectilinear post-productionediting to the sub-composition.
 16. A method for 360-degree videopost-production in accordance with claim 1, wherein the videopost-production editing comprises rectilinear post-production editing.17. A method for 360-degree video post-production in accordance withclaim 1, further comprising outputting the 360-degree compositiontogether with the one or more layers to a display.
 18. A method for360-degree video post-production in accordance with claim 17, whereinthe display is a 360-degree display.
 19. A system for 360-degree videopost-production, comprising: memory; an output interface devicecomprising a display; an input interface device; a processor operablycoupled to the memory, the output interface device and the inputinterface device, the processor for executing instructions stored in thememory which, when executed, causes the processor to perform: displayingan interface screen for a 360-degree composition on an output interfacedevice comprising a display, the interface screen configured to displayan omni directional camera viewer that is configured to display arectilinear view of the 360-degree composition as a consequence toorientation navigating through the omni directional camera viewer;identifying a point of view comprising receiving selection through theinterface screen of a rectilinear view displayed in the interfacescreen; applying video post-production editing to the identified pointof view, comprising tracking the identified point of view to identifytracking information comprising one or more point cloud layers and atracked camera layer; linking the one or more point cloud layers to thetracked camera layer; aligning the one or more point cloud layers in the360-degree composition with the identified point of view; and saving oneor more layers, comprising the point cloud layers and the tracked cameralayer, to memory.
 20. A non-transitory computer readable mediumcomprising instructions for 360-degree video post-production andexecutable by one or more processors, the instructions comprisinginstructions to perform: displaying an interface screen for a 360-degreecomposition on an output interface device comprising a display, theinterface screen configured to display an omni directional camera viewerthat is configured to display a rectilinear view of the 360-degreecomposition as a consequence to orientation navigating through the omnidirectional camera viewer; identifying a point of view comprisingreceiving selection through the interface screen of a rectilinear viewdisplayed in the interface screen; applying video post-productionediting to the identified point of view, comprising tracking theidentified point of view to identify tracking information comprising oneor more point cloud layers and a tracked camera layer; linking the oneor more point cloud layers to the tracked camera layer; aligning the oneor more point cloud layers in the 360-degree composition with theidentified point of view; and saving one or more layers, comprising thepoint cloud layers and the tracked camera layer, to memory.